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A significant portion of the Orion pre-PDR design effort has focused on balancing mass 
with performance. High level performance metrics include abort success rates, lunar 
surface coverage, landing accuracy and touchdown loads. These metrics may be converted 
to parameters that affect mass, such as ballast for stabilizing the abort vehicle, propellant to 
achieve increased lunar coverage or extended missions, or ballast to increase the lift-to-drag 
ratio to improve entry and landing performance. The Orion Flight Dynamics team was 
tasked to perform analyses to evaluate many of these trades. These analyses not only 
provide insight into the physics of each particular trade but, in aggregate, they illustrate the 
processes used by Orion to balance performance and mass margins, and thereby make 
design decisions. Lessons learned can be gleaned from a review of these studies which will 
be useful to other spacecraft system designers. These lessons fall into several categories, 
including: appropriate application of Monte Carlo analysis in design trades, managing 

margin in a highly mass-constrained environment, and the use of requirements to balance 
margin between subsystems and components. This paper provides a review of some of the 
trades and analyses conducted by the Flight Dynamics team, as well as systems engineering 
lessons learned. 


I. Introduction 

T he Orion Crew Exploration Vehicle (CEV) consists of three major components (Figure 1): a Crew Module 
(CM), Service Module (SM) and a Launch Abort System (LAS). The requirement for Orion to carry four crew 
members to and from the Moon has resulted in a larger, heavier CM than the Apollo design. The capsule diameter is 
16.5 ft, nearly 4 ft larger than the Apollo capsule with 2.5 times the 
internal volume. This larger capsule requires a larger, more powerful 
LAS to achieve required separation during launch aborts. The SM, on the 
other hand is smaller than its Apollo counterpart, since the task of 
inserting the vehicle into lunar orbit falls to the lunar lander propulsion 
system, rather than the SM engine, as in Apollo. Even with the smaller 
service module though, mass has proven to be a critical design parameter 
for Orion. The mass of the Orion CM affects abort system performance, 
landing loads, parachute sizing, launch vehicle performance and trans- 
lunar injection (TLI) performance. The mass of the SM drives the range 
of lunar surface sites that are achievable at a given Earth-Moon geometry as well as launch and TLI performance. 

A significant portion of the Orion pre-PDR design effort has focused on balancing mass with performance. High 
level performance metrics include abort success rates, lunar surface coverage, mission duration limits, landing 
opportunity rates and landing accuracy. These metrics may be converted to parameters that affect mass, such as 
ballast for stabilizing the abort vehicle, propellant to achieve increased lunar coverage or extended missions, or 
ballast to increase the lift-to-drag ratio to improve entry and landing performance. The Orion Flight Dynamics team 
was tasked to perform analyses to evaluate many of these trades. These analyses not only provide insight into the 
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physics of each particular trade but, in aggregate, they illustrate the processes used by Orion to balance performance 
and mass margins, and thereby make design decisions. Lessons learned can be gleaned from a review of these 
studies which will be useful to other spacecraft system designers. These lessons fall into several categories, 
including: appropriate application of Monte Carlo analysis in design trades, managing margin in a highly mass- 
constrained environment, and the use of requirements to balance margin between subsystems. This paper provides a 
review of some of the trades conducted by the Flight Dynamics team. 

The following sections describe selected trade studies whose results quantified the sensitivities of vehicle 
performance to mass. Trade study descriptions are divided into ascent abort, orbit, and entry sections. The ascent 
abort section covers a trade on the stability of the abort vehicle versus ballast mass. Orbit trades selected are “fail 
safe” Moon-to-Earth transfer maneuvers and Lunar coverage studies. Entry studies include entry performance 
versus ballast mass, and active roll control performance under chutes to orient the vehicle at touchdown. Each trade 
study contains a summary of the results in terms of performance margin and sensitivity of performance to mass. 

After the trade study discussions we use the margin and sensitivity data to provide a notional example of the 
margin balancing that is performed at the Orion project level. The example highlights many of the issues that must 
be dealt with by Orion systems engineers and provides a conceptual framework for using the study information 

The final sections present lessons learned from the conduct of the studies and from the application of their results 
in vehicle-level integration. These include lessons on simulation and Monte Carlo analysis - from avoiding traps 
during the analysis to interpreting the results correctly. 

Definition of Margin and Sensitivity 

To aid in better defining margin and sensitivity for the rest of this paper, it is useful to define a relative 
performance metric. This metric is the performance requirement , divided by the measured performance. Using 
landing accuracy as an example, if a study shows that 3 times the standard deviation (3a) of landing accuracy for a 
particular set of parameters is 4 nautical miles, and the required landing accuracy (3a) is 5 nm, then the relative 
performance metric is 1.2- so this relative metric equals one when the performance equals the requirement. Margin 
then, is the relative performance metric minus one. In the landing example, margin is 0.2 or 20%. 

Sensitivity is normally thought of as the partial derivative of the performance metric to a cost variable. The cost 
variable used here is mass, but other cost metrics are possible, including monetary cost. In the landing example, if 
adding 100 lb of ballast to the CM increases landing accuracy by 0.2 nautical miles, then the sensitivity of accuracy 
to mass is 0.2/100 or 0.002 nm/lb. This type of sensitivity however does not allow the systems engineer to trade 
different types of performance. Which is worth more, for example, landing accuracy or lunar surface coverage? 

The measure of “importance” of a particular performance metric is the requirement. So for comparison to other 
performance metrics, we want the sensitivity of margin with respect to the required performance. In the landing 
example, if the additional mass improves landing accuracy to 3.8 nm, then margin is increased to 22% and the 
sensitivity of the margin to mass (in percent) is 1 00 times the performance sensitivity divided by the requirement, or 
0.04. We will use this type of sensitivity (margin to mass) in our examples derived from the trade studies below. 

II. Ascent and Abort Trade Study Example 

During ascent, Orion may abort from a failing, or under-performing launch vehicle in two ways. During first 
stage flight, a Launch Abort System (LAS) is provided to remove the Crew Module (CM) capsule from a booster 
experiencing catastrophic failure. Shortly after first stage burnout, the LAS is jettisoned and the responsibility for 
separation from the launch vehicle falls to the Service Module (SM). Second stage booster failures are generally 
considered more benign that first stage failures. This section describes one of many trades conducted to help design 
the LAS. 

A. Stability vs Ballast 

The Orion Launch Abort System (LAS) consists of three rocket motors housed in a tower-fairing assembly 
(Figure 2). The abort motor provides the thrust to remove the Launch Abort Vehicle (LAV) - the combined LAS 
and capsule - from the malfunctioning booster. The jettision motor detaches the LAS from the capsule after it has 
been re-oriented to a heat shield forward attitude during the abort. The attitude control motor (ACM) is a solid 
rocket engine whose thrust is directed through a ring of eight “pintle” valves. The net thrust vector is controlled by 
opening or closing valves in the ring. The role of the ACM is to stabilize and control the LAV during the rocket 
motor burn, and to re-orient the LAV to the heat-shield forward attitude for LAS jettison and subsequent chute 
deploy.The Orion abort system is required to prove a 95% success rate averaged over the ascent time period. A 
further design objective is to eliminate specific periods during the ascent with low success probabilities. 

2 

American Institute of Aeronautics and Astronautics 





Attitude Control/* I 
Motor Nozzle RiusJ 

Jettison Motor 
Nozzles 


Abort Motor 
Nozzles 

Center of 
Pressure v 
DuringAboit^ 


Aerodynamic 
Moment Arm 


Motor Bum 


Center of 
Gravity 


®] 

& 


One of the challenges of controlling the LAV is that the abort motor 
thrust plumes move the aerodynamic center of pressure ahead of the 
center of gravity making the LAV dynamically unstable during the 
abort burn (Figure 3). In addition to the normal challenges associated 
with un-stable vehicles, the forward center of pressure increases the 
aerodynamic moment arm for pitch and yaw. This means that large 
aerodynamic torques may occur when the vehicle is displaced from the 
trim angle of attack. 

The degree to which the abort motor plumes affect the center of 
pressure is difficult to determine precisely. The resulting uncertainty in 
pitch moment torque 

means that under certain *bort Motor Plumes 

conditions the 

aerodynamic pitch 

moment may exceed the 
available torque from the 
ACM control motor. 

Should this occur the 
vehicle will tumble out 
of control. 

The combination of 

large aerodynamic uncertainties together with increases in the CM mass 
allocation, resulted in the vehicle tumbling during a significant number of simulated abort cases - especially when 
aborts occurred around Mach 1. To correct these tumbles, weight or ballast may be added in the top portion of the 
LAS tower. This moves the center of gravity forward, reducing the aerodynamic moment arm of Figure 2. 

To determine the effect of ballast on abort success, a Monte Carlo simulation study was conducted with varying 
amounts of ballast. Figure 4 summarizes the tumbling failure rates for several quantities of ballast. Aborts were 
initiated at altitudes between zero (the launch pad) and around 65000 ft - the point at which the LAS is jettisoned 
after first stage separation. Parameters varied in the Monte Carlo included aerodynamic parameters, mass properties 
and rocket motor parameters. 

The figure shows that with no ballast (blue curve) there are altitude regions in the ascent profile for which the 
probability of abort failure due to tumbling are relatively high - more than 35% at the peak and greater than 5% for 
a large section of the ascent. The improvements in success rate are clear from the reduced tumbling cases associated 
with even the relatively “small” 660 lb nose-cone ballast. 

The results of this study provided Orion management and systems decision makers data on the success rate as a 
function of ballast mass. As we shall see, the mass data may be used as a common “currency” to trade first stage 
abort success performance against other 



Figure 3. Abort Motor Plumes 


Figure 2. Launch Abort Vehicle 


spacecraft attributes which enhace crew safety, 
improve capability or increase mission success 
rates. When considering the ballast mass 
numbers against other sources of mass on the 
Orion vehicle, bear in mind that, since the LAS 
is jettisoned after first stage, mass on the LAS is 
not as critical to ascent performance as other 
vehicle mass which is fully lifted into orbit. A 
rule of thumb calculation for comparing LAS 
mass to CM or SM mass is that one pound of 
LAS mass affects ascent performance about 1/8 
as much as one pound of CM or SM mass. 

In order to balance margins across the 
vehicle design, we must quantify the margin on 
vehicle-level requirements. That presents some 
difficulty with the success rate requirements that 
apply to the LAS. For example, consider the 
LAV requirement of 95% success. Even if the 


Histogram: Percent Tumbling vs, Initial Altitude 



Figure 4. Effect of Ballast on Tumbling 
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vehicle is successful 100% of the time, the margin on this requirement is only 5% so standard margins of 30% at 
PDR and 10% at CDR are not acheiveable. Further, this limit on maximum margin precludes margin balancing 
since no amount of ballast assures 100% success. For these reasons, we have taken the approach of redefining 
margin based on a fictitious “underlying performance parameter.” Failure is presumed to occur when that 
parameter exceeds a limit. The details of this process are provided in the appendix since, although they are 
interesting, they are not germain to the overall process of trading LAS abort performance. 

Although the 95% time-averaged success requirement does not preclude high failure rates at particular points 
along the profile, the flight dynamics team has the design goal of limiting failure rates along the profile. As of this 
writing, the vehicle level trades are not complete to determine how much, if any ballast will be allocated. However, 
for the purposes of the margin balancing discussion below, we will assume that the amount of ballast used should be 
sufficient to keep success rates above 80% at all points along the profile. This assumption also simplifies the 
determination of margin and sensitivity, since we now need only consider the data at the peak failure rate - a little 
more than 30,000 ft in Figure 4. 

Table 1 summarizes the LAV representative performance requirements, margins and sensitivities gleaned from 
this and similar Monte Carlo studies. Note that with no ballast the LAV has negative margin on the success 
requirement. 


Table 1. Margin and Sensitivity for LAV Ballast Trade 


Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

Success vs. Ballast Mass 

80% Success at any altitude 

-19% 

0.23 


Of course, other parameters affect performance as well as ballast. Expected performance may be improved by 
reducing aerodynamic uncertainties through flight or wind-tunnel tests, by improving the control design, or by any 
number of other techniques. But as Orion approaches CDR the requirements, control design and major 
environmental models will become more stable, and final decisions will be made to allocate mass according to the 
remaining vehicle-level “knobs” such as ballast and propellant. 


III. Orbit Trades 

This section presents two trade studies conducted to relate Orion performance to SM propellant mass. The first 
study looked at the techniques used in targeting and executing main engine burns at the moon. This study was used 
to determine the assumptions that should be used in generating propellant budgets for lunar missions. The second 
study determined available lunar surface coverage as a function of SM propellant. 

A. Fail Safe Trans Earth Injection 

During the lunar mission, after the Altair lander has returned the crew to Orion in lunar orbit, the Altair ascent 
stage is jettisioned, and the Orion SM performs the series of burns at the Moon to initiate the orbital transfer back to 
Earth. A typical three burn, trans-Earth injection (TEI) series is depicted in Figure 5. TEI-1 transfers Orion from a 
near circular low lunar orbit (LLO) to a highly elliptical orbit (yellow arc). Near the apoapse of this orbit, TEI-2 is 
executed to place the vehicle in a lunar orbit plane that corresponds to the plane of the desired Earth-relative velocity 
for the return trip. TEI-3 is then executed to accelerate away from the Moon and enter an Earth-relative orbit with 

the desired periapse for Entry. On the return 
trip, trajectory correction maneuvers (TCMs) are 
executed to correct dispersions from the ideal 
TEI sequence and to precisely target the entry 
interface conditions. 

During the TEI sequence, two types of 
failures can cause the spacecraft to impact the 
Moon: 1) failure to execute a burn - which can 
cause impact if the current orbit has a perigee 
below the lunar surface or, 2) an engine failure 
during a burn - which can cause impact if the 
resulting orbit has a perigee below the surface. 

Figure 5. Trans-Earth Injection (TEH Seauence 
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The first of these failures - failure to execute - results in impact if the previous burn left the spacecraft in an 
orbit with a low perigee. This is a function of how the previous burn is targeted and can occur if, for example, the 

burn had a radial component. Experience has 
shown that a fuel optimal burn sequence can 
result in orbits with unsafe periapse. Figure 6 
shows an example in which TEI-1 is executed 
to target the optimal TEI-2 position - 
resulting in an orbit that impacts the lunar 
surface if TEI-2 is not executed. As the 
figure shows, from the time that TEI-2 fails to 
execute (t=0) to lunar surface impact is 12.2 
hours. For this particular failure then, some 
time remains to correct the problem, prevent 
lunar impact, and re-target the return. 

This situation may be contrasted with that 
of Figure 7. In this example, TEI-2 is 

executed normaly, but the optimal solution leaves Orion in a lunar impact orbit. This solution relies on TEI-3 
executing properly to prevent collision with the Moon, and - as the figure shows - only leaves about 12 minutes to 
correct a problem once the failure is discovered at TEI-3 execution. 

The collisions depicted in Figures 6 
and 7 may be prevented by adopting a 
policy constraining the targeting of all 
burns to result in periapse above a 
prescribed altitude. This policy is 
referred to here as “fail safe targetting.” 
The second type of failure 
considered is an engine failure during 
the burn. Figure 8 is a conceptual 
diagram of the velocity change executed 
at TEI-2. In this example, the velocity 
direction is changed by 90° at or near 
apoapse, resulting in an orbit whose 
plane is consistent with the orbit plane 
needed for Earth return. The red traces in the figure represent two candidate burn execution strategies. On the left 
the “chord burn” is executed in the direction to most efficiently and directly change the velocity from the intial 
orbital velocity to the final desired. However, if the burn should stop for any reason along the red trace, the 
resulting velocity magnitude is less 
than the initial and final velocities. 

This smaller velocity magnitude may 
result in an orbit with lower periapse 
which impacts the moon. 

In contrast, the fail safe burn is 
executed so that the resulting velocity 
magnitude always results in a safe 
orbit - at the cost of the extra 
propellant corresponding to the longer 
red arc. 

The fail safe TEI trade study 
quantified the propellant mass costs 
for both fail safe targeting and fail safe execution. Propellant numbers are omitted here since spacecraft 
performance may be sensitive, but the study recommandations may be understoon from the above discussion. They 
were: 

• TEI-1 should be targeted to prevent low periapsis since the fuel penalty was always very small 

• TEI-2 execution is not required to be fail safe since many hours are available to correct the resulting 
orbit, and since auxiliary thrusters are available to back up the Orion main engine (OME) 

• TEI-2 is targeted to guarantee a safe periapsis since little time remains after a TEI-3 failure 

5 

American Institute of Aeronautics and Astronautics 


Chord Burn 


Fail-Safe Burn 




Vfinal 



Figure 8. Fail Safe Burn Geometry 



Figure 7. Impact Resulting from Failed TEI-3 Execution 
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Figure 6. Impact Resulting from Failed Burn Execution 




This trade study did not quantify margin with respect to a requirement. Rather, it resulted in a policy for use in 
other design trades, such as the following lunar suface coverage trade study. 


B. Lunar Geographic and Temporal Coverage 

The Orion concept of operations allows for two types of lunar missions. Lunar outpost missions are intended to 
service a lunar base at the Moon’s south pole. During outpost missions, Orion loiters unmanned in LLO for up to 
six months. Lunar sortie missions are intended to allow exploration of any portion of the lunar surface and are much 
shorter than lunar outpost missions - lasting around 18 days under nominal conditions with a week on the lunar 
surface. 




In order to size Orion propellant 
loads and tank capacities, it is 
useful to relate the amount of the 
lunar surface that can be reached to 
propellant mass. To this end, it is 
useful to define two types of lunar 
coverage: geographic coverage, 

and temporal coverage. 
Geographic coverage is the amount 
of the Moon’s surface that may be 
reached at a given launch epoch 
while temporal coverage is the 
percentage of time (over the 19 
year Metonic cycle) that a 
particular site can be reached. 

Given prescribed launch vehicle 
Figure 9. Return Trajectories from Assorted Landing Site and lander ca p acitieS) the reac hable 

portions of the Moon are 

determined by Orion’s capacity to change it’s lunar orbit plane to pass over the landing site, and then reach an 
appropriate entry interface condition via the three burn TEI sequence described above. This capacity depends on the 
total inertial velocity change, Av, that Orion can produce, which in turn depends on the specific impulse (I sp ) of the 
engines, and the amount of onboard propellant. 

Figure 9 shows the TEI sequences and return trajectories for several landing sites at constant latitude and varying 
longitude around the Moon. From the figure, it is clear that the TEI burn sequence associated with a particular site 
has a large effect on the total Av required to return from that site. So if the propellant load is smaller than that 
required for the worst of these 
trajectories, then some percentage of 
these sites will not be available for 
exploration at a given epoch. 

Figure 10 summarizes the results 
of a study conducted to determine 
site availability as a function of 
propellant load. The study 
developed mission designs 
(sequences of burns) for a grid of 
landing sites around the Moon 
across a set of departure epochs that 
spanned the Metonic cycle which 
governs Earth-Moon geometry. The 
study consisted of nearly 1 billion 
site-epoch combinations. 

The horizontal axis of the figure 
is the onboard propellant in Kg and 
the vertical axis shows the 
percentage of the lunar surface that 
may be reached given a particular 
propellant load and a desired 


100 

90 


5900 6100 6300 6500 6700 6300 7100 7300 7500 7700 7900 B1QQ 8300 B5QQ 870 

Orion Undispersed Useable Propellant (kg) 

Figure 10. Geographic and Temporal Coverage as a Function of 
Propellant Load 
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amount of temporal coverage. The curves on the graph are the temporal coverage available at a given combination 
of propellant load and surface coverage. 

To read the graph, start with a desired propellant load - say the usable propellant load indicated by the thick red 
vertical line. Follow the line up to a desired temporal coverage curve - say 90% temporal coverage. The 
intersection of the red propellant line with the 90% curve occurs at about 80% surface coverage. This means that 
with around 7400 Kg of useable propellant, Orion can reach 80% of the Moon’s surface 90% of the time. For the 
same propellant load, Orion can reach about 65% of the surface 100% of the time. 

The solid red boundary in the figure is the total propellant required to ideally actuate the mission Av assuming no 
targeting, navigation or control errors. Propellant must be added to the ideal value to account for these errors, or 
dispersions, in order to fly the real mission. This additional propellant is indicated by the thin red boundary in the 
figure. Margin may also be added to the total propellant load to account for uncertainties or contingencies. The thin 
red boundary may be thought of as the “design fuel load” since it represents the amount of fuel that is accounted for 
when trading propellant mass against other forms of mass (strucuture, ballast, heat shield, crew supplies, etc). The 
design fuel load may be viewed as a requirement which may be adjusted by systems engineers to balance margin. 

For our purposes, we will consider the 100% temporal coverage of 65% of the surface to be the propellant 
defining requirement. Recall that the corresponding propellant also provides considerably more surface coverage 
during more favorable ephocs. Table 2 contains notional values (actual values may be sensitive) of the margin that 
the design propellant load contains. The sensitivity of surface coverage margin to propellant mass is also provided. 


Table 2. Margin and Sensitivity for Lunar Surface Coverage Trade 


Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

Surface Coverage vs. Propellant 
Mass 

65% of lunar surface acheiveable on 
100% of launch epochs 

20% 

0.12 


Finally, we note that the thin blue boundary in the Figure 10 represents the total propellant available in the SM 
tanks. Tank capacity exceeds the design fuel load to provide operational flexibility and future performance growth 
should other mass opportunities manifest themselves. Note that the dispersions must also be subtracted from the 
tank sizing capacity to get the “useable” fuel that may be ideally converted to Av. 

IV. Entry Trades 

Two trades from the entry flight phase further illustrate the methods used to relate performance to mass. The 
first relates the capsule lift-to-drag ratio to ballast mass, and the second trades touchdown landing performance with 
structural mass. 


A. Entry Performance vs Ballast Mass 

After entering the atmosphere, the Orion CM seeks a “trim” attitude that balances aerodynamic torques. Figure 
1 1 diagrams some the parameters which determine the pitch trim angle. In the figure, a, or angle of attack is the 

angle between the relative air flow and the 
longitudinal axis of the vehicle (blue dashed 
line). At small angles of attack, the 
aerodynamic center of pressure on the CM is 
close to the center of the heat shield. When 
the center of gravity is offset from the 
longitudinal axis, a torque is generated about 
the CG that equals the aerodynamic force 
times the moment arm from the longitudinal 
axis to the CG. This torque causes the 
capsule to rotate in pitch until this torque is 
balanced by the restoring torques that occur 
from the symmetrical cone’s tilt. At steady state then, the magnitude of the trim angle of attack depends on the CG 
offset from the longitudinal axis. This value tends to be fairly constant through hypersonic and supersonic flight. 



Figure 11. Lift Generation in a Symmetric Capsule 
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The tilt of the symmetrical cone-shaped capsule at trim creates lift in the direction of the CG offset. As depicted 
in the figure, the CG offset is constructed to create lift in the general direction of the crew’s feet. This upside down 
posture, allows the crew to view the horizon out of the capsule windows. 

At speeds above the transonic region, the capsule is stable at trim angle of attack, and aerodynamic torques resist 
perturbations in angle of attack away from trim. For this reason, pitch and yaw are not directly controlled once in 
the atmosphere, instead the reaction control thrusters are used to damp pitch and yaw rates to reduce oscillation 
about trim. In fact, once in the atmosphere, pitch and yaw control torques are insufficient to overcome aerodynamic 
torques, so controlling to an attitude far from trim is not possible. 

The bank angle of the capsule on the other hand, is controlled - primarily by the roll RCS thrusters. Control of 
the bank angle determines the orientation of the lift vector about the relative airflow. The figure depicts zero bank 
or “lift up.” By modulating bank, the direction of lift may be controlled to determine the capsule sink rate. Higher 
sink rates cause the capsule to encounter denser air more quickly, thereby increasing drag and shortening the 
downrange flown. In general then, a constant zero bank profile minimizes sink rate and maximizes range, while a 
constant 180° bank, or “lift down” profile minimizes range. Guidance may command bank angles between zero and 
180° to achieve the range of the landing site. The resulting heading changes are small, and are compensated for by 
periodically reversing the bank direction to keep the heading azimuth within preset limits of the landing site bearing. 

At zero angle of attack, lift is zero and all 
aerodynamic force is opposite the air flow in the 
direction of drag. So the amount of lift generated 
depends on the CG offset. A particular offset determines 
the trim angle of attack, which in turn determines the 
ratio of lift to drag, or L/D ( “L over D”). L/D is 
approximately constant from entry enterface to near the 
drogue deploy point. Larger values of L/D expand the 
available “footprint” of possible landing sites from a 
given entry interface condition. 

Since L/D is determined by the CG offset, the CM 
designers achieve as much offset as possible by 
packaging heavy spacecraft components in the desired 
offset direction. For Orion, this has not resulted in CG 
offsets which meet L/D requirements, so additional 
ballast must be added on the edge of the capsule cone to 
create more offset. A design trade must then be 
performed between the ballast mass, and the 
performance associated with increased L/D. 

To determine the minimum L/D, a series of Monte Carlo analyses were conducted at various entry geometries 
for both lunar and low Earth orbit (LEO) missions. Figure 12 is an example of the results for lunar return with a 
constant targeted flight path angle. Note that miss distance is well within the requirement for L/D values above 0.23 
and the sensitivity of miss remains small with decreasing L/D until suddenly increasing. This “cliff’ behavior is due 
to the fact that the spacecraft can no longer fly the required range to the site with insufficient lift. 

From these data, the requirement for L/D may be stated as: “The mean lift-to-drag ratio minus three times the 
standard deviation of the lift-to-drag ratio shall be greater than or equal to 0.23.” 

Table 3 contains notional values of the L/D margin that the design ballast mass provides. The sensitivity of L/D 
to ballast mass is a fairly straightforward calculation and is detailed in the appendix? 


Table 3. Margin and Sensitivity for L/D versus Ballast Trade 


Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

L/D vs Ballast Mass 

Mean L/D - 3 *ct of L/D >= 0.23 
(lunar) 

15% 

0.05 
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Figure 12. Sensitivity of Miss Distance to L/D 


It should be noted that a maximum L/D requirement is also enforced, since high L/D values (and their associated 
trim angles of attack) can cause violations of thermal constraints. However, the minimum L/D value minimizes 
ballast mass, so it is of most interest for this discussion. More information about L/D constraints and their 
associated statistical limitations on CG placement, see reference XXX. 
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B. Roll Orientation Trades at Touchdown 


Desired Orientation 


The Orion crew module is required to have 
enough habitable volume to carry four astronauts to 
the Moon with associated life support systems and 
expendables. The resultant capsule diameter is 
16.5 ft, nearly 4 ft larger than the Apollo capsule 
with 2.5 times the internal volume. The larger 
capsule diameter presents structural design 
problems, especially during water landings in high 
sea states. Landing impact attenuation systems Figure 13. Desired Landing Orientation 

designed to protect the crew from, one-chute-out 

touchdown loads require increased mass and volume to handle worst-case combinations of sink rate, horizontal 
velocity, and wave impact geometry. 

In order to mitigate the risk of structural failure or crew load exceedance the orientation of the capsule is 
controlled at touchdown to provide the best direction of impact for the attenuation system and to minimize the 
probability of impacting a wave in a “belly flop” orientation. Figure 13 illustrates the desired orientation with 
respect to the surface-relative velocity at impact (V re i). Since the horizontal component of the relative velocity is 
due primarily to wind, changes in wind direction in the final 1500 ft or so prior to touchdown result in changes in 
relative velocity which must be tracked by the on-board control system. 

The ability to respond to changes in horizontal velocity is a function of velocity measurement accuracy and 
control response time. Accurate measurement of small, wind-driven horizontal velocities requires GPS. Even with 
GPS accuracy, however, the accuracy depends on velocity magnitude, which at steady state equals wind speed. So 
high winds, resulting in large horizontal velocities are easily measured, while low winds make velocity direction 
determination difficult. Fortunately, roll orientation at low relative speed is not critical for crew loading or structural 
deformation. 

Control response is also a factor in achieving the desired orientation. When wind direction changes within the 
last few feet of the descent, the control system fires roll thrusters to turn the vehicle toward the new direction. The 
change in orientation may cause the main chute risers to twist, creating a back-torque disturbance with which the 
controller must contend. Further the controller is required to have a high success rate with the loss of one of the two 
redundant strings of CM thrusters, so control authority may be reduced in the worst case. Fortunately, in high-wind 

situations, wind direction variability is reduced and the wind 
direction is more likely to persist. 

Figure 14 is a plot of the touchdown roll error against 
horizontal velocity magnitude for a Monte Carlo study which 
modeled wind variations, chute dynamic parameter variations, 
and navigation errors. The red boundary is the requirement for 
roll control accuracy, and reflects the maximum allowable 
value of three times the standard deviation of the error. Note 
that when the horizontal velocity is less than 10 ft/s, no roll 
control is required. At speeds greater than 10 ft/s, the 
allowable error is decreased. This reflects the fact that both 
crew loads and worst-case structural loads are worse at high 
horizontal impact speeds. Also note that the shape of the 
“cloud” of blue error markers reflects the improved 
navigation accuracy and reduced wind variability at higher 

I loriztmtal Velocity ( ft/s) wind spee( J s 

Figure 14. Roll Heading Error versus The red requirement boundary in the figure may also be 

Horizontal Velocity viewed as the system level design “knob” to allow systems 

engineers to trade the risk of landing in an undesireable 
orientation, against the mass required to make the structure and crew loads attenuation system robust to larger errors 
at higher gound speeds. Since the shape of the requirement line is complex we will choose a single driving “choke 
point” to determine margins and sensitivities. This point is at 40 ft/s in the figure and the margin is determined by 
comparing the boundary error at this point (±20°) with three times the standard deviation of the error in the vicinity 
of the point (for simplicity, we shall assume that the mean error is zero). 



Undesired Orientation 
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Here the mass of interest is the mechanism and structural mass required to limit crew loads and maintain 
structural integrity. The sensitivity of interest then is the sensitivity of the heading error at 40 ft/sec to structural and 
mechanism mass. Notional values of the margin and sensitivity are provided in Table 4. 


Table 4. Margin and Sensitivity for Headin: 

g Error Trade 

Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

Heading error performance 
Structural Mass 

vs. 

3*g of heading error < 20 deg at 40 
ft/sec 

15% 

0.33 


V. Vehicle Design Application and Lessons Learned 

The studies summarized above provided information to the Orion SEIT team and Orion mana gement to make 
vehicle-level design decisions which are still in process at the time of this writing. This section looks at a process 
for combining the study results to make design decisions, and provides lessons learned during the conduct of these 
and other studies. It is important for the analyst to bear in mind the purpose of the analysis since the information 
must be presented to management in a clear manner, and should include the underlying physics (at a high level), the 
applicable requirements, the margin on those requirements and the sensitivity of that margin to mass. 

A. Margin Balancing 

For the purposes of illustrating some of the elements of a decision making process, we have simplified the study 
results somewhat so that a classical margin balancing technique may be used to allocate mass between vehicle 
components. After discussing these process elements, considerations for real world application will be covered. 

Table 5 summarizes the data and results of four of the studies presented above. These four studies had attributes 
that lend themselves well to mathematical margin balancing, namely they relate mass to a required performance or 
safety metric, and their results provided the margins and sensitivities required to balance margin. In the following 
discussions, we refer to mass as the “currency” used to balance margin. Since we have a fixed amount of currency 
(mass) in Orion, the systems engineering decision is how to allocate, or “spend” the currency to optimize 
performance. 


Table 5. Margin Balancing for Mass Allocation 


Study 

Trade 

Requirement/ 
Design Objective 

Margin 

Sensitivity 

(%/lb) 

Am for 
Margin 
Balance (lb) 

Balanced 

Margin 

LAS Ballast 

Success vs. 
Ballast Mass 

80% Success at any 
altitude 

-19% 

0.23 

+134.9 

12.0% 

Lunar Surface 
Coverage 

Surface Coverage 
vs. Propellant 
Mass 

65% of lunar surface 
acheiveable on 
100% of launch 
epochs 

20% 

0.12 

-66.4 

12.0% 

Entry 

Performance 

L/D vs Ballast 
Mass 

Mean L/D -3*aof 
L/D >= 0.23 (lunar) 

15% 

0.05 

-59.5 

12.0% 

Touchdown 

Heading 

Control 

Heading error 
performance vs. 
Structural Mass 

3*g of heading 
error < 20 deg 

15% 

0.33 

-9.0 

12.0% 


Recall that the ballast margin for the LAS is not lifted all the way to orbit as discussed above, so the ballast 
mass delta shown in the table is multiplied approximately by 8 to arrive at the actual ballast to be housed in the LAS 
- or around 1000 lb. 

Once the system level designer has the information of margin and sensitivity to each requirement with respect to 
a common currency, he should balance the margin. This may be accomplished by one of these methods: 

1. Modify the requirements: It is always a good idea to revisit the relative importance of the various 
requirements during margin balancing. How important is the abort success rate compared to Lunar 
surface coverage given abort probabilities and mission objectives? These kinds of decisions must be 
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made at a project level and the role of the Flight Dynamics team and other engineering organizations is 
to provide accurate, relevant and digestable data to aid in the decision. 

2. Re-allocate the currency: A mathematical example of this process is illustrated in the right two columns 
of the table and the equations which were applied are derieved in the appendix from the straight forward 
assumptions that the margin must be balanced with zero change in mass and after balancing, the 
margins are equal on all requirements. 

3. Apply management reserve: The assumption that the mass change must be zero assumes that no mass 
growth is allowed. As the design matures, management may opt to release mass margin in the form of 
reduced weight growth allowance or reduced management reserve. This will allow improved 
performance across the flight regimes in the table, but the margin balancing principals remain the same. 

4. A combination of the above: Systems engineers may elect to modify requirements, re-allocate mass and 
apply reserve. 

The Orion systems engineering problem is much more complex that the example presented. There are many 
more performance metrics, precise sensitivities are often difficult to determine, and mass on the SM is not 
equivalent to mass on the CM since they drive different performance metrics. In practice, systems engineers 
combine classical margin balancing with engineering judgement, but systems engineers should always know their 
margins, understand sensitivities, and ensure that no margins stand out. 

The last of these is the crux of the vehicle-level systems engineers role. The role of the flight dyanamics analyst, 
on the other hand, is to provide data on margins and sensitivities. The next section discusses some lessons gleaned 
from these and other flight dyanamics studies which were conducted with the intent of supplying data to make 
vehicle-level decisions. 

B. Use of Time Domain Simulation and Monte Carlo Analysis in Trade Studies 

All of the trade studies described above used time domain simulation to produce the trade results. Time domain 
simulations techniques have become more and more popular over the last few decades to generate data for vehicle 
design. Monte Carlo analysis is often used to derive statistical correlations and sensitivities as shown above. This 
approach has some benefits, including: 

• The process of simulation model development and integration often provides experience and insight to 
the developer into the governing physics and performance drivers in applicable flight phases (but this 
may not always be true for the analyst who may not have been involved in model development) 

• Modern computer systems are capable of handling hundreds of parameter variations over thousands of 
simulation runs. These can provide accurate Monte Carlo statistics over the parameter ranges used for 
the models used 

On Orion, Monte Carlo analysis will be used extensively for requirements verification, but it has also been used 
for requirements development, sensitivity analysis, and vehicle-level design. Design recommendations gleaned 
from these applications include: 

• Understand the underlying physics 

• Use Monte Carlo (or other statistical approaches) sparingly in requirements allocation or error 
budgeting 

• Understand when worst-case analysis is conservative and when it is appropriate 

• Avoid pitfalls in Monte Carlo analysis for requirements verification 

The following paragraphs details these recommendations. 

Recommendation: Understand the underlying physics 

The first recommendation is well known, but bears repeating. When new, inexperienced engineers join a 
project, they are often assigned simulation analysis tasks. These “turn the crank” jobs can allow engineers to begin 
producing results soon after joining an organization. However, it takes discipline on the part of supervisory 
personnel to make sure that the simulations are configured correctly, that the statistical attributes of Monte Carlo 
parameters are properly set, that output statistics are correctly interpreted, and that the analyst develops a knowledge 
of the underlying physics. We will discuss some of the statistical aspects below, but the system knowledge can only 
be gained by accompanying the simulation study with theoretical analysis. One rule of thumb: “never be surprised 
by a simulation result.” That is to say that the simulation or Monte Carlo analysis should confirm what the analyst 
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expects or should refine the analysts’ estimate. When this is not the case, the analyst must understand why the 
results do not match expectations. The sim can be wrong! 

Recommendation: Use Monte Carlo sparingly in requirements allocation or error budgeting 

Figure 15 is a simple diagram of the Monte Carlo process which will aid in discussing lessons learned. In the 
process, the statistical properties of the input parameters are defined, then random values are selected for each of the 
parameters, and a simulation (or other process) is executed using these values. The resulting parameters of interest 
are recorded, and the process repeated many times. The statistical properties of the output provides information that 
may be used to decompose requirements, serve as an input for other design processes or to verify existing 
requirements. 

As an example of requirements decomposition, consider the heading error example in the roll orientation trade 
above. For this example the requirement on the GNC substystem was “three times the standard deviation of the 
heading error (assuming zero mean) at 40 ft/sec groundspeed shall be less than 20°. ” One of the jobs of the GNC 

system designer is to decompose this requirement into 
navigation accuracy and control error allocations. In this 
example the total error is just the sum of the navigation error 
and the control error. Here, control error includes error due 
to disturbances - such as wind gusts wind shear. 

One familiar approach to allocating error to navigation 
Figure 15. Monte Carlo Process and control is the worst case approach. The worst case may 

be defined as a hard limit, or it may be defined by a project 
policy such as “design to three sigma.” Either way, the total error is the sum of the worst case navigation and 
control errors. If the total error meets the requirement, then a portion of the margin may be allocated to the lower 
level requirements in proportion to their contribution. This method is the most conservative method of performing 
allocation since total error is highly unlikely to exceed the sum of the worst case errors. It is also the simplest 
method. 

If the total error does not meet the requirement, then the systems engineer may elect to petition for a more 
generous requirement (at a structural mass cost), try to improve navigation and or control performance (perhaps at a 
cost of more expensive hardware or more complex software) - or he may elect to use a less conservative allocation 
process. One method would be to assume that the total error distribution is a statistical combination of the input 
distributions. If both navigation and control error are random with Gaussian distribution, then the distribution of the 
sum of the errors is Gaussian with standard deviation equal to the root-mean-square (RMS) of the inputs. 

If the Gaussian statistical assumptions are not applicable to the inputs, or if the relationship between inputs and 
outputs is complex, then, Monte Carlo analysis is sometimes used to relate the statistical properties of the inputs to 
the outputs. However, if Monte Carlo is used, two pitfalls must be avoided: 

• The output distribution must be close enough to Gaussian to meet the intent of the requirement 

• Margin must be applied during requirements allocation 

The later pitfall is the most common and causes the most difficulty. If the lower level requirements (navigation 
and control error specifications in the example) are simply left at their expected “3a” values, and if there is 
insufficient margin at the next level (total heading error in the example), then any change in applicable parameters or 
assumptions may cause an increase in navigation or control variances, which may in turn cause an exceedance of the 
heading error requirement, which may necessitate redefinition of the requirement, which can affect structural mass 
and ultimately require re-allocation of mass at the vehicle level. In our example, if the landing site changes, and the 
wind shear statistics worsen, then the control variances will increase, and the effect can propagate up the 
requirements chain. To prevent this, margin should be consciously allocated early through all requirements levels. 

Recommendation: Understand when worst-case analysis is conservative and when it is appropriate 

As mentioned above, worst case analysis is a simple approach to requirement decomposition and includes a level 
of conservatism or “built in” margin. But analysts often confuse the low probabilityof a worst case combination 
with conservatism. To illustrate this, consider Figure 16 - an abstraction of the input parameter space. The 
horizontal axis represents the value of an input parameter, “pi” and the vertical axis another parameter, “p2.” The 
mean values of pi and p2 are shown by the cross at center, while the red box represents the range of values between 
mean minus 3a and mean plus 3a. The green ellipse represents the region in the pi, p2 parameter space for which a 
requirement is met. 
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Region of input space 



Pi 


Figure 16. Parameter Box 


The common argument for using a statistical approach to reduce 
conservatism is to note that the probablitly of the pi, p2 pair existing 
near the comers of the box is low. [Include equation here. I think it’s 
(l-99.73) A n]. As the number of parameters increases, the probability of 
parameters combining at the corners of the box in a “worst on worst” 
combination rapidly approaches zero. So assessing to the corners of the 
box that cause requirements exceedance may be unnecessarily 
conservative. 

While it is true that worst case analysis assumes a low probability 
condition, the degree to which it is conservative depends on the relative 
sensitivity of the output to the parameters. The left diagram of Figure 
17 shows a case where pi is a “driving” parameter and the output is 
relatively insensitive to p2. In this case, evaluating the corner of the box 
does not add a significant amount of 
conservatism - even though the pi, p2 pair at 
the corner is highly improbable. And the same 
is tme for the right hand diagram, in which the 
output is not highly sensitive to either 
parameter in the context of their variances. 

It is often the case that a system response is 
driven largely by one or two parameters, so, 
depending on the amount of margin available, 
and the relative sensitivity of the output to the 
other parameters, a worst case, or “corners of 
the box” approach to requirement 

decomposition is often the simplest and most robust method. 

This worst case approach was used by the Orion flight dynamics team in defining the requirements for CG offset 
as discussed in the L/D section above. For missions to ISS, the CG offset was derived by assuming that 
aerodynamic parameters were at their worst values (when uniformly distributed) or their 3a values when Gaussian. 
This allowed a simple definition for a CG box with adequate margin for uncertainties and operational variations in 
CG. For the Lunar mission, however, margin was much more constrained - largely by a higher minimum L/D 
requirement, so a more sophisticated statistical approach was taken, as described in reference XXX. 



Figure 17. Driving Parameters 


Recommendation: Avoid pitfalls in Monte Carlo analysis for requirements verification 


As the project approaches CDR, the requirements allocation process is largely complete, and the focus shifts 
toward proving requirements compliance via pre-CDR verification. During this phase, simulation analysis using 
higher fidelity models is an appropriate method for assessing requirements compliance. However, several common 
pitfalls should be avoided: 

1 . Set parameters according to bounding requirements - not expected values 

2. Margin is determined from output results - not input variations 

3. ... however, input variations can and should be used to determine robustness - find out where it breaks 

4. TBS 

Conclusions 

TBS 


Appendix 


A. Margin Balancing Equations 

Once margin and sensitivity data are available for each requirement, the equations to re-allocate the independent 
cost variable (often mass, or monetary cost, or a combination) may be derived as follows: 

The change in margin for the i th requirement, A is added to the existing margin to achieve the balanced margin 
which we desire to be constant for all requirements: 
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The change in margin may be replaced by the change in cost variable, C times the sensitivity of margin to the 
cost variable: 

(i) 

so 

and 

( 2 ) 

Now, when re-allocating cost, the net cost change between all n requirements must be zero: 

= 0 
so 

= 0 

This expression contains the constant balanced margin for each requirement, M b , and the know values for the 
margins and sensitivities for each requirement. To solve for M b , the common denominator for each of the terms in 
the sum is multiplied by each sum: 


Define: 


so 


so the desired balanced margin value is: 


( 3 ) 


and the changes in cost variable and margin for each requirement are may be determined from 1, and 2 above. 


B. Margin for Statistical Success Requirements 

The approach taken in this exercise was to presume that abort failure is the result of an “underlying performance 
parameter” that exceeds a limit. We further assume that the underlying parameter has a Gaussian distribution of 
mean zero and a standard deviation that causes the limit to be exceeded 5% of the time. This standard deviation 
then is the required standard deviation of the parameter. We then compute the standard deviation that causes the 
parameter to exceed the limit at the observed percentage. Margin is then computed as the difference between these 
standard deviations. 

TBS. 
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A significant portion of the Orion pre-PDR design effort has focused on balancing mass 
with performance. High level performance metrics include abort success rates, lunar 
surface coverage, landing accuracy and touchdown loads. These metrics may be converted 
to parameters that affect mass, such as ballast for stabilizing the abort vehicle, propellant to 
achieve increased lunar coverage or extended missions, or ballast to increase the lift-to-drag 
ratio to improve entry and landing performance. The Orion Flight Dynamics team was 
tasked to perform analyses to evaluate many of these trades. These analyses not only 
provide insight into the physics of each particular trade but, in aggregate, they illustrate the 
processes used by Orion to balance performance and mass margins, and thereby make 
design decisions. Lessons learned can be gleaned from a review of these studies which will 
be useful to other spacecraft system designers. These lessons fall into several categories, 
including: appropriate application of Monte Carlo analysis in design trades, managing 

margin in a highly mass-constrained environment, and the use of requirements to balance 
margin between subsystems and components. This paper provides a review of some of the 
trades and analyses conducted by the Flight Dynamics team, as well as systems engineering 
lessons learned. 


Nomenclature 


ACM 

= Attitude Control Motor 

RMS 

= Root Mean Square 

CEV 

= Crew Exploration Vehicle 

SM 

= Service Module 

CG 

= Center of Gravity 

TCM 

= T raj ectory Correction Maneuver 

CM 

= Crew Module 

TEI 

= Trans Earth Injection 

GN&C 

= Guidance Navigation and Control 

TLI 

= T rans-lunar Inj ection 

GPS 

= Global Positioning System 



LAS 

= Launch Abort System 

AV 

= Change in inertial velocity 

LAV 

= Launch Abort Vehicle 

dsp 

= Specific Impulse 

LLO 

= Low Lunar Orbit 

L/D 

= Lift to Drag Ratio 

OME 

= Orion Main Engine 

C r el 

= Earth Relative Velocity 


I. Introduction 

T he Orion Crew Exploration Vehicle (CEV) consists of three major components (Figure 1): a Crew Module 
(CM), Service Module (SM) and a Launch Abort System (LAS). The requirement for Orion to carry four crew 
members to and from the Moon has resulted in a larger, heavier CM than the Apollo design. The capsule diameter is 
16.5 ft, nearly 4 ft larger than the Apollo capsule with 2.5 times the internal volume. This larger capsule requires a 
larger, more powerful LAS to achieve required separation during launch aborts. The SM, on the other hand is 
smaller than its Apollo counterpart, since the task of inserting the vehicle into lunar orbit falls to the lunar lander 
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propulsion system, rather than the SM engine, as in Apollo. Even with the 
smaller service module though, mass has proven to be a critical design 
parameter for Orion. The mass of the Orion CM affects abort system 
performance, landing loads, parachute sizing, launch vehicle performance 
and trans-lunar injection (TLI) performance. The mass of the SM drives 
the range of lunar surface sites that are achievable at a given Earth-Moon 
geometry as well as launch and TLI performance. 

A significant portion of the Orion pre-PDR design effort has focused 
on balancing mass with performance. High level performance metrics 
include abort success rates, lunar surface coverage, mission duration 
limits, landing opportunity rates and landing accuracy. These metrics may be converted to parameters that affect 
mass, such as ballast for stabilizing the abort vehicle, propellant to achieve increased lunar coverage or extended 
missions, or ballast to increase the lift-to-drag ratio to improve entry and landing performance. The Orion Flight 
Dynamics team was tasked to perform analyses to evaluate many of these trades. These analyses not only provide 
insight into the physics of each particular trade but, in aggregate, they illustrate the processes used by Orion to 
balance performance and mass margins, and thereby make design decisions. Lessons learned can be gleaned from a 
review of these studies which will be useful to other spacecraft system designers. These lessons fall into several 
categories, including: appropriate application of Monte Carlo analysis in design trades, managing margin in a 
highly mass-constrained environment, and the use of requirements to balance margin between subsystems. This 
paper provides a review of some of the trades conducted by the Flight Dynamics team. 

The following sections describe selected trade studies whose results quantified the sensitivities of vehicle 
performance to mass. Trade study descriptions are divided into ascent abort, orbit, and entry sections. The ascent 
abort section covers a trade on the stability of the abort vehicle versus ballast mass. Orbit trades selected are “fail 
safe” Moon-to-Earth transfer maneuvers and Lunar coverage studies. Entry studies include entry performance 
versus ballast mass, and active roll control performance under chutes to orient the vehicle at touchdown. Each trade 
study contains a summary of the results in terms of performance margin and sensitivity of performance to mass. 

After the trade study discussions we use the margin and sensitivity data to provide a notional example of the 
margin balancing that is performed at the Orion project level. The example highlights many of the issues that must 
be dealt with by Orion systems engineers and provides a conceptual framework for using the study information 

The final sections present lessons learned from the conduct of the studies and from the application of their results 
in vehicle-level integration. These include lessons on simulation and Monte Carlo analysis - from avoiding traps 
during the analysis to interpreting the results correctly. 

Definition of Margin and Sensitivity 

To aid in better defining margin and sensitivity for the rest of this paper, it is useful to define a relative 
performance metric. This metric is the measured performance, divided by the performance requirement. Using 
landing accuracy as an example, if a study shows that 3 times the standard deviation (3 a) of landing accuracy for a 
particular set of parameters is 4 nautical miles, and the required landing accuracy (3a) is 5 nm, then the relative 
performance metric is 0.8 - so this relative metric equals one when the performance equals the requirement. Margin 
then, is one minus the relative performance metric. In the landing example, margin is 0.2 or 20%. 

Sensitivity is normally thought of as the partial derivative of the performance metric to a cost variable. The cost 
variable used here is mass, but other cost metrics are possible, including monetary cost. In the landing example, if 
adding 100 lb of ballast to the CM increases landing accuracy by 0.2 nautical miles, then the sensitivity of accuracy 
to mass is 0.2/100 or 0.002 nm/lb. This type of sensitivity however does not allow the systems engineer to trade 
different types of performance. Which is worth more, for example, landing accuracy or lunar surface coverage? 

The measure of “importance” of a particular performance metric is the requirement. So for comparison to other 
performance metrics, we want the sensitivity of margin with respect to the required performance. In the landing 
example, if the additional mass improves landing accuracy to 3.8 nm, then margin is increased to 22% and the 
sensitivity of the margin to mass (in percent) is 1 00 times the performance sensitivity divided by the requirement, or 
0.04. We will use this type of sensitivity (margin to mass) in our examples derived from the trade studies below. 

II. Ascent and Abort Trade Study Example 

During ascent, Orion may abort from a failing, or under-performing launch vehicle in two ways. During first 
stage flight, a Launch Abort System (LAS) is provided to remove the Crew Module (CM) capsule from a booster 
experiencing catastrophic failure. Shortly after first stage burnout, the LAS is jettisoned and the responsibility for 
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separation from the launch vehicle falls to the Service Module (SM). Second stage booster failures are generally 
considered more benign that first stage failures. This section describes one of many trades conducted to help design 
the LAS. 

A. Stability vs Ballast 

The Orion Launch Abort System (LAS) consists of three rocket motors housed in a tower-fairing assembly 
(Figure 2). The abort motor provides the thrust to remove the Launch Abort Vehicle (LAV) - the combined LAS 
and capsule - from the malfunctioning booster. The jettision motor detaches the LAS from the capsule after it has 

been re-oriented to a heat shield forward attitude during the abort. The 
attitude control motor (ACM) is a solid rocket engine whose thrust is 
directed through a ring of eight “pintle” valves. The net thrust vector is 
controlled by opening or closing valves in the ring. The role of the ACM 
is to stabilize and control the LAV during the rocket motor burn, and to re- 
orient the LAV to the heat-shield forward attitude for LAS jettison and 
subsequent chute deploy. The Orion abort system is required to prove a 
95% success rate averaged over the ascent time period. A further design 
objective is to eliminate specific periods during the ascent with low 
success probabilities. 

One of the challenges of controlling the LAV is that the abort motor 
thrust plumes move the aerodynamic center of pressure ahead of the center 
of gravity making the LAV dynamically unstable during the abort burn 
(Figure 3). In addition to the normal challenges associated with un-stable 
vehicles, the forward center of pressure increases the aerodynamic moment 
arm for pitch and yaw. This means that large aerodynamic torques may 
occur when the vehicle is displaced from the trim angle of attack. 

The degree to which the abort motor plumes affect the center of 
pressure is difficult to determine precisely. The resulting uncertainty in 
pitch moment torque means that under certain conditions the aerodynamic 
pitch moment may exceed the 
available torque from the 
ACM control motor. Should 
this occur the vehicle will tumble out of control. 

The combination of large aerodynamic uncertainties together with 
increases in the CM mass allocation, resulted in the vehicle tumbling 
during a significant number of simulated abort cases - especially when 
aborts occurred around Mach 1. To correct these tumbles, weight or 
ballast may be added in the top portion of the LAS tower. This moves 
the center of gravity forward, reducing the aerodynamic moment arm of 
Figure 2. 

To determine the effect of ballast on abort success, a Monte Carlo simulation study was conducted with varying 
amounts of ballast. Figure 4 summarizes the tumbling failure rates for several quantities of ballast. Aborts were 
initiated at altitudes between zero (the launch pad) and around 65000 ft - the point at which the LAS is jettisoned 
after first stage separation. Parameters varied in the Monte Carlo included aerodynamic parameters, mass properties 
and rocket motor parameters. 

The figure shows that with no ballast (blue curve) there are altitude regions in the ascent profile for which the 
probability of abort failure due to tumbling are relatively high - more than 35% at the peak and greater than 5% for 
a large section of the ascent. The improvements in success rate are clear from the reduced tumbling cases associated 
with even the relatively “small” 660 lb nose-cone ballast. 

The results of this study provided Orion management and systems decision makers data on the success rate as a 
function of ballast mass. As we shall see, the mass data may be used as a common “currency” to trade first stage 
abort success performance against other spacecraft attributes which enhace crew safety, improve capability or 
increase mission success rates. When considering the ballast mass numbers against other sources of mass on the 
Orion vehicle, bear in mind that, since the LAS is jettisoned after first stage, mass on the LAS is not as critical to 
ascent performance as other vehicle mass which is fully lifted into orbit. A rule of thumb calculation for comparing 
LAS mass to CM or SM mass is that one pound of LAS mass affects ascent performance about 1/8 as much as one 
pound of CM or SM mass. 
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Figure 3. Abort Motor Plumes 
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Figure 2. Launch Abort Vehicle 



In order to balance margins across the vehicle 
design, we must quantify the margin on vehicle- 
level requirements. That presents some difficulty 
with the success rate requirements that apply to 
the LAS. For example, consider the LAV 
requirement of 95% success. Even if the vehicle 
is successful 100% of the time, the margin on this 
requirement is only 5% so standard margins of 
30% at PDR and 10% at CDR are not 
acheiveable. Further, this limit on maximum 
margin precludes margin balancing since no 
amount of ballast assures 100% success. For 
these reasons, we have taken the approach of 
redefining margin based on a fictitious 
“underlying performance parameter.” Failure is 
presumed to occur when that parameter exceeds a 
limit. The details of this process are provided in 
the appendix since, although they are interesting, 
they are not germain to the overall process of trading LAS abort performance. 

Although the 95% time-averaged success requirement does not preclude high failure rates at particular points 
along the profile, the flight dynamics team has the design goal of limiting failure rates along the profile. As of this 
writing, the vehicle level trades are not complete to determine how much, if any ballast will be allocated. However, 
for the purposes of the margin balancing discussion below, we will assume that the amount of ballast used should be 
sufficient to keep success rates above 80% at all points along the profile. This assumption also simplifies the 
determination of margin and sensitivity, since we now need only consider the data at the peak failure rate - a little 
more than 30,000 ft in Figure 4. 

Table 1 summarizes the LAV representative performance requirements, margins and sensitivities gleaned from 
this and similar Monte Carlo studies. Note that with no ballast the LAV has negative margin on the success 
requirement. 


Table 1. Margin and Sensitivity for LAV Ballast Trade 


Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

Success vs. Ballast Mass 

80% Success at any altitude 

-19% 

0.23 


Histogram: Percent Tumbling vs Initial Altitude 



Figure 4. Effect of Ballast on Tumbling 


Of course, other parameters affect performance as well as ballast. Expected performance may be improved by 
reducing aerodynamic uncertainties through flight or wind-tunnel tests, by improving the control design, or by any 
number of other techniques. But as Orion approaches CDR the requirements, control design and major 
environmental models will become more stable, and final decisions will be made to allocate mass according to the 
remaining vehicle-level “knobs” such as ballast and propellant. 


III. Orbit Trades 

This section presents two trade studies conducted to relate Orion performance to SM propellant mass. The first 
study looked at the techniques used in targeting and executing main engine burns at the moon. This study was used 
to determine the assumptions that should be used in generating propellant budgets for lunar missions. The second 
study determined available lunar surface coverage as a function of SM propellant. 

A. Fail Safe Trans Earth Injection 

During the lunar mission, after the Altair lander has returned the crew to Orion in lunar orbit, the Altair ascent 
stage is jettisioned, and the Orion SM performs the series of burns at the Moon to initiate the orbital transfer back to 
Earth. A typical three burn, trans-Earth injection (TEI) series is depicted in Figure 5. TEI-1 transfers Orion from a 
near circular low lunar orbit (LLO) to a highly elliptical orbit (yellow arc). Near the apoapse of this orbit, TEI-2 is 
executed to place the vehicle in a lunar orbit plane that corresponds to the plane of the desired Earth-relative velocity 
for the return trip. TEI-3 is then executed to accelerate away from the Moon and enter an Earth-relative orbit with 
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the desired periapse for Entry. On the return trip, trajectory correction maneuvers (TCMs) are executed to correct 
dispersions from the ideal TEI sequence and to precisely target the entry interface conditions. 

During the TEI sequence, two types of 
failures can cause the spacecraft to impact the 
Moon: 1) failure to execute a burn - which 
can cause impact if the current orbit has a 
perigee below the lunar surface or, 2) an 
engine failure during a burn - which can cause 
impact if the resulting orbit has a perigee 
below the surface. 

The first of these failures - failure to 
execute - results in impact if the previous 
burn left the spacecraft in an orbit with a low 
perigee. This is a function of how the 
previous burn is targeted and can occur if, for 
example, the burn had a radial component. 

Experience has shown that a fuel optimal burn 
sequence can result in orbits with unsafe 
periapse. Figure 6 shows an example in which 
TEI-1 is executed to target the optimal TEI-2 
position - resulting in an orbit that impacts the 
lunar surface if TEI-2 is not executed. As the 
figure shows, from the time that TEI-2 fails to 
execute (t=0) to lunar surface impact is 12.2 
hours. For this particular failure then, some 
time remains to correct the problem, prevent 
lunar impact, and re-target the return. 

This situation may be contrasted with that 
of Figure 7. In this example, TEI-2 is executed 
normaly, but the optimal solution leaves Orion 
in a lunar impact orbit. This solution relies on 
TEI-3 executing properly to prevent collision 
with the Moon, and - as the figure shows - 
only leaves about 12 minutes to correct a 
problem once the failure is discovered at TEI-3 
execution. 

The collisions depicted in Figures 6 and 7 
may be prevented by adopting a policy 
constraining the targeting of all burns to result 
in periapse above a prescribed altitude. This 
policy is referred to here as “fail safe 
targetting.” 

The second type of failure considered is an engine failure during the burn. Figure 8 is a conceptual diagram of 
the velocity change executed at TEI-2. In this example, the velocity direction is changed by 90° at or near apoapse, 
resulting in an orbit whose plane is consistent with the orbit plane needed for Earth return. The red traces in the 

figure represent two candidate burn execution 
strategies. On the left the “chord burn” is executed in 
the direction to most efficiently and directly change the 
velocity from the intial orbital velocity to the final 
desired. However, if the burn should stop for any 
reason along the red trace, the resulting velocity 
magnitude is less than the initial and final velocities. 
This smaller velocity magnitude may result in an orbit 
with lower periapse which impacts the moon. 

In contrast, the fail safe burn is executed so that the 


Chord Burn 


Fail-Safe Bum 




Figure 8. Fail Safe Burn Geometry 



Figure 5. Trans-Earth Injection (TEI) Sequence 
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Figure 6. Impact Resulting from Failed Burn Execution 



Figure 7. Impact Resulting from Failed TEI-3 Execution 
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resulting velocity magnitude always results in a safe orbit - at the cost of the extra propellant corresponding to the 
longer red arc. 

The fail safe TEI trade study quantified the propellant mass costs for both fail safe targeting and fail safe 
execution. Propellant numbers are omitted here since spacecraft performance may be sensitive, but the study 
recommendations may be understood from the above discussion. They were: 

• TEI-1 should be targeted to prevent low periapsis since the fuel penalty was always very small 

• TEI-2 execution is not required to be fail safe since many hours are available to correct the resulting 
orbit, and since auxiliary thrusters are available to back up the Orion main engine (OME) 

• TEI-2 is targeted to guarantee a safe periapsis since little time remains after a TEI-3 failure 

This trade study did not quantify margin with respect to a requirement. Rather, it resulted in a policy for use in 
other design trades, such as the following lunar suface coverage trade study. 


B. Lunar Geographic and Temporal Coverage 

The Orion concept of operations allows for two types of lunar missions. Lunar outpost missions are intended to 
service a lunar base at the Moon’s south pole. During outpost missions, Orion loiters unmanned in LLO for up to 
six months. Lunar sortie missions are intended to allow exploration of any portion of the lunar surface and are much 
shorter than lunar outpost missions - lasting around 18 days under nominal conditions with a week on the lunar 
surface. 

In order to size Orion propellant loads 
and tank capacities, it is useful to relate the 
amount of the lunar surface that can be 
reached to propellant mass. To this end, it is 
useful to define two types of lunar coverage: 
geographic coverage, and temporal 
coverage. Geographic coverage is the 
amount of the Moon’s surface that may be 
reached at a given launch epoch while 
temporal coverage is the percentage of time 
(over the 19 year Metonic cycle) that a 
particular site can be reached. Given 
prescribed launch vehicle and lander 
capacities, the reachable portions of the 
Moon are determined by Orion’s capacity to 
change it’s lunar orbit plane to pass over the 
landing site, and then reach an appropriate entry interface condition via the three burn TEI sequence described 
above. This capacity depends on the total inertial velocity change, Av, that Orion can produce, which in turn 
depends on the specific impulse (I sp ) of the engines, and the amount of onboard propellant. 

Figure 9 shows the TEI sequences and return trajectories from several landing sites at constant latitude and 
varying longitude around the Moon. From the figure, it is clear that the TEI burn sequence associated with a 
particular site has a large effect on the total Av required to return from that site. So if the propellant load is smaller 
than that required for the worst of these trajectories, then some percentage of these sites will not be available for 
exploration at a given epoch. 

Figure 10 summarizes the results of a study conducted to determine site availability as a function of propellant 
load. The study developed mission designs (sequences of burns) for a grid of landing sites around the Moon across 
a set of departure epochs that spanned the Metonic cycle which governs Earth-Moon geometry. The study consisted 
of nearly 1 billion site-epoch combinations. 
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Figure 9. Return Trajectories from Assorted Landing Site 
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The horizontal axis of the figure 
is the onboard propellant in 
kilograms and the vertical axis 
shows the percentage of the lunar 
surface that may be reached given a 
particular propellant load and a 
desired amount of temporal 
coverage. The curves on the graph 
are the temporal coverage available 
at a given combination of propellant 
load and surface coverage. 

To read the graph, start with a 
desired propellant load - say the 
usable propellant load indicated by 
the thick red vertical line. Follow 
the line up to a desired temporal 
coverage curve - say 90% temporal 
coverage. The intersection of the 
red propellant line with the 90% 
curve occurs at about 80% surface 
coverage. This means that with 
around 7400 Kg of useable 
propellant, Orion can reach 80% of 
the Moon’s surface 90% of the time, 
of the time. 

The solid red boundary in the figure is the total propellant required to ideally actuate the mission Av assuming no 
targeting, navigation or control errors. Propellant must be added to the ideal value to account for these errors, or 
dispersions, in order to fly the real mission. This additional propellant is indicated by the thin red boundary in the 
figure. Margin may also be added to the total propellant load to account for uncertainties or contingencies. The thin 
red boundary may be thought of as the “design fuel load” since it represents the amount of fuel that is accounted for 
when trading propellant mass against other forms of mass (strucuture, ballast, heat shield, crew supplies, etc). The 
design fuel load may be viewed as a requirement which may be adjusted by systems engineers to balance margin. 

For our purposes, we will consider the 100% temporal coverage of 65% of the surface to be the propellant 
defining requirement. Recall that the corresponding propellant also provides considerably more surface coverage 
during more favorable ephocs. Table 2 contains notional values (actual values may be sensitive) of the margin that 
the design propellant load contains. The sensitivity of surface coverage margin to propellant mass is also provided. 



Orion Undispersed Useable Propellant (kg) 

Figure 10. Geographic and Temporal Coverage as a Function of 
Propellant Load 

For the same propellant load, Orion can reach about 65% of the surface 100% 


Table 2. Margin and Sensitivity for Lunar Surface Coverage Trade 


Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

Surface Coverage vs. Propellant 
Mass 

65% of lunar surface acheiveable on 
100% of launch epochs 

20% 

0.12 


Finally, we note that the thin blue boundary in the Figure 10 represents the total propellant available in the SM 
tanks. Tank capacity exceeds the design fuel load to provide operational flexibility and future performance growth 
should other mass opportunities manifest themselves. Note that the dispersions must also be subtracted from the 
tank sizing capacity to get the “useable” fuel that may be ideally converted to Av. 

IV. Entry Trades 

Two trades from the entry flight phase further illustrate the methods used to relate performance to mass. The 
first relates the capsule lift-to-drag ratio to ballast mass, and the second trades touchdown landing performance with 
structural mass. 

A. Entry Performance vs Ballast Mass 
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Figure 11. Lift Generation in a Symmetric Capsule 


After entering the atmosphere, the Orion CM seeks a “trim” attitude that balances aerodynamic torques. Figure 
1 1 diagrams some of the parameters which determine the pitch trim angle. In the figure, a, or angle of attack is the 

angle between the relative air flow and the 
longitudinal axis of the vehicle (blue dashed 
line). At small angles of attack, the 
aerodynamic center of pressure on the CM is 
close to the center of the heat shield. When 
the center of gravity (CG) is offset from the 
longitudinal axis, a torque is generated about 
the CG that equals the aerodynamic force 
times the moment arm from the longitudinal 
axis to the CG. This torque causes the 
capsule to rotate in pitch until this torque is 
balanced by the restoring torques that occur 
from the symmetrical cone’s tilt. At steady state then, the magnitude of the trim angle of attack depends on the CG 
offset from the longitudinal axis. This value tends to be fairly constant through hypersonic and supersonic flight. 

The tilt of the symmetrical cone-shaped capsule at trim creates lift in the direction of the CG offset. As depicted 
in the figure, the CG offset is constructed to create lift in the general direction of the crew’s feet. This upside down 
posture allows the crew to view the horizon out of the capsule windows. 

At speeds above the transonic region, the capsule is stable at trim angle of attack, and aerodynamic torques resist 
perturbations in angle of attack away from trim. For this reason, pitch and yaw are not directly controlled once in 
the atmosphere, instead the reaction control thrusters are used to damp pitch and yaw rates to reduce oscillation 
about trim. In fact, once in the atmosphere, pitch and yaw control torques are insufficient to overcome aerodynamic 
torques, so controlling to an attitude far from trim is not possible. 

The bank angle of the capsule on the other hand is controlled - primarily by the roll thrusters. Control of the 
bank angle determines the orientation of the lift vector about the relative airflow. The figure depicts zero bank or 
“lift up.” By modulating bank, the direction of lift may be controlled to determine the capsule sink rate. Higher 
sink rates cause the capsule to encounter denser air more quickly, thereby increasing drag and shortening the 
downrange flown. In general then, a constant zero bank profile minimizes sink rate and maximizes range, while a 
constant 180° bank, or “lift down” profile minimizes range. Guidance may command bank angles between zero and 
1 80° to achieve the range of the landing site. The resulting heading changes are small, and are compensated for by 
periodically reversing the bank direction to keep the 
heading azimuth within preset limits of the landing site 
bearing. 

At zero angle of attack, lift is zero and all 
aerodynamic force is opposite the air flow in the direction 
of drag. So the amount of lift generated depends on the 
CG offset. A particular offset determines the trim angle 
of attack, which in turn determines the ratio of lift to drag, 
or L/D ( “L over D”). Larger values of L/D expand the 
available “footprint” of possible landing sites from a 
given entry interface condition. 

Since L/D is determined by the CG offset, the CM 
designers achieve as much offset as possible by packaging 
heavy spacecraft components in the desired offset 
direction. For Orion, this has not resulted in CG offsets 
which meet L/D requirements, so additional ballast must 
be added on the edge of the capsule cone to create more 
offset. A design trade must then be performed between 
the ballast mass, and the performance associated with increased L/D. 

To determine the minimum L/D, a series of Monte Carlo analyses were conducted at various entry geometries 
for both lunar and low Earth orbit (LEO) missions. Figure 12 is an example of the results for lunar return with a 
constant targeted flight path angle. Note that miss distance is well within the requirement for L/D values above 0.23 
and the sensitivity of miss remains small with decreasing L/D until suddenly increasing. This “cliff’ behavior is due 
to the fact that the spacecraft can no longer fly the required range to the site due to insufficient lift. 


Miss Distance over Range 


- Constraint 
| Mean 

| Mean+3sigma 
] Maximum 


0.23 


0.27 


L/D 


0.30 


0.33 


Figure 12. Sensitivity of Miss Distance to L/D 
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From these data, the requirement for L/D may 
be stated as: “The mean lift-to-drag ratio minus 

three times the standard deviation of the lift-to-drag 
ratio shall be greater than or equal to 0.23 at Mach 
25.” 

Table 3 contains notional values of the L/D 
margin that the design ballast mass provides. The 
sensitivity of L/D to ballast mass is a fairly 
straightforward calculation and is detailed in the 
appendix? 

Table 3. Margin and Sensitivity for L/D versus 


Ballast Trade 


Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

L/D vs Ballast Mass 

Mean L/D - 3*o of L/D >= 0.23 
(lunar) 

15% 

0.05 


Undesired Orientation Desired Orientation 



Figure 13. Desired Landing Orientation 


It should be noted that a maximum L/D requirement is also enforced, since high L/D values (and their associated 
trim angles of attack) can cause violations of thermal constraints. However, the minimum L/D value minimizes 
ballast mass, so it is of most interest for this discussion. For more information about L/D constraints and their 
associated statistical limitations on CG placement, see reference XXX. 


B. Roll Orientation Trades at Touchdown 


As noted above, the Orion crew module is required to have enough habitable volume to carry four astronauts to 
the Moon with associated life support systems and expendables. The resultant capsule diameter is 16.5 ft, nearly 4 ft 
larger than the Apollo capsule with 2.5 times the internal volume. The larger capsule diameter presents structural 
design problems, especially during water landings in high sea states. Landing impact attenuation systems designed 

to protect the crew from, one-chute-out touchdown loads 
require increased mass and volume to handle worst-case 
combinations of sink rate, horizontal velocity, and wave 
impact geometry. 

In order to mitigate the risk of structural failure or crew 
load exceedance the orientation of the capsule is controlled 
at touchdown to provide the best direction of impact for the 
attenuation system and to minimize the probability of 
impacting a wave in a “belly flop” orientation. Figure 13 
illustrates the desired orientation with respect to the 
surface-relative velocity at impact (V re i). Since the 
horizontal component of the relative velocity is due 
primarily to wind, changes in wind direction in the final 
1500 ft or so prior to touchdown result in changes in 
relative velocity which must be tracked by the on-board 
control system. 

The ability to respond to changes in horizontal velocity 
is a function of velocity measurement accuracy and control 
response time. Accurate measurement of small, wind- 
driven horizontal velocities requires GPS. Even with GPS accuracy, however, the accuracy depends on velocity 
magnitude, which at steady state equals wind speed. So high winds resulting in large horizontal velocities are easily 
measured, while low winds make velocity direction determination difficult. Fortunately, roll orientation at low 
relative speed is not critical for crew loading or structural deformation. 

Control response is also a factor in achieving the desired orientation. When wind direction changes within the 
last few feet of the descent, the control system fires roll thrusters to turn the vehicle toward the new direction. The 
change in orientation may cause the main chute risers to twist, creating a back-torque disturbance with which the 



I lorizontal Velocity (ft/s) 


Figure 14. Roll Heading Error versus 
Horizontal Velocity 
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controller must contend. Further the controller is required to have a high success rate with the loss of one of the two 
redundant strings of CM thrusters, so control authority may be reduced in the worst case. Fortunately, in high-wind 
situations, wind direction variability is reduced and the wind direction is more likely to persist. 

Figure 14 is a plot of the touchdown roll error against horizontal velocity magnitude for a Monte Carlo study 
which modeled wind variations, chute dynamic parameter variations, and navigation errors. The red boundary is the 
requirement for roll control accuracy, and reflects the maximum allowable value of three times the standard 
deviation of the error. Note that when the horizontal velocity is less than 10 ft/s, no roll control is required. At 
speeds greater than 1 0 ft/s, the allowable error is decreased. This reflects the fact that both crew loads and worst- 
case structural loads are worse at high horizontal impact speeds. Also note that the shape of the “cloud” of blue 
error markers reflects the improved navigation accuracy and reduced wind variability at higher wind speeds. 

The red requirement boundary in the figure may also be viewed as the system level design “knob” to allow 
systems engineers to trade the risk of landing in an undesireable orientation against the mass required to make the 
structure and crew loads attenuation system robust to larger errors at higher gound speeds. Since the shape of the 
requirement line is complex, a single driving “choke point” is chosen to determine margins and sensitivities. This 
point is at 40 ft/s in the figure and the margin is determined by comparing the boundary error at this point (±20°) 
with three times the standard deviation of the error in the vicinity of the point (for simplicity, we shall assume that 
the mean error is zero). 

Here the mass of interest is the mechanism and structural mass required to limit crew loads and maintain 
structural integrity. The sensitivity of interest then is the sensitivity of the heading error at 40 ft/sec to structural and 
mechanism mass. Notional values of the margin and sensitivity are provided in Table 4. 


Table 4. Margin and Sensitivity for Heading Error Trade 


Trade 

Requirement/ Design Objective 

Margin 

Sensitivity (%/lb) 

Heading error performance vs. 
Structural Mass 

3 *g of heading error < 20 deg at 40 
ft/sec 

15% 

0.33 


V. Vehicle Design Application and Lessons Learned 

The studies summarized above provided information to the Orion SEIT team and Orion mana gement to make 
vehicle-level design decisions which are still in process at the time of this writing. This section looks at a process 
for combining the study results to make design decisions, and provides lessons learned during the conduct of these 
and other studies. It is important for the analyst to bear in mind the purpose of the analysis since the information 
must be presented to management in a clear manner, and should include the underlying physics (at a high level), the 
applicable requirements, the margin on those requirements and the sensitivity of that margin to mass. 

A. Margin Balancing 

For the purposes of illustrating some of the elements of a decision making process, we have simplified the study 
results somewhat so that a classical margin balancing technique may be used to allocate mass between vehicle 
components. After discussing these process elements, considerations for real world application will be covered. 

Table 5 summarizes the data and results of four of the studies presented above. These four studies had attributes 
that lend themselves well to mathematical margin balancing, namely they relate mass to a required performance or 
safety metric, and their results provided the margins and sensitivities required to balance margin. In the following 
discussions, we refer to mass as the “currency” used to balance margin. Since we have a fixed amount of currency 
(mass) in Orion, the systems engineering decision is how to allocate, or “spend” the currency to optimize 
performance. 
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Table 5. Margin Balancing for Mass Allocation 


Study 

Trade 

Requirement/ 
Design Objective 

Margin 

Sensitivity 

(%/lb) 

Am for 
Margin 
Balance (lb) 

Balanced 

Margin 

LAS Ballast 

Success vs. 
Ballast Mass 

80% Success at any 
altitude 

-19% 

0.23 

+134.9 

12.0% 

Lunar Surface 
Coverage 

Surface Coverage 
vs. Propellant 
Mass 

65% of lunar surface 
acheiveable on 
100% of launch 
epochs 

20% 

0.12 

-66.4 

12.0% 

Entry 

Performance 

L/D vs Ballast 
Mass 

Mean L/D -3*aof 
L/D >= 0.23 (lunar) 

15% 

0.05 

-59.5 

12.0% 

Touchdown 

Heading 

Control 

Heading error 
performance vs. 
Structural Mass 

3*g of heading 
error < 20 deg 

15% 

0.33 

-9.0 

12.0% 


Recall that the ballast margin for the LAS is not lifted all the way to orbit as discussed above, so the ballast 
mass delta shown in the table is multiplied approximately by 8 to arrive at the actual ballast to be housed in the LAS 
- or around 1000 lb. 

Once the system level designer has the information of margin and sensitivity to each requirement with respect to 
a common currency, the margin should be balanced. This may be accomplished by one of these methods: 

1. Modify the requirements : It is always a good idea to revisit the relative importance of the various 
requirements during margin balancing. How important is the abort success rate compared to Lunar 
surface coverage given abort probabilities and mission objectives? These kinds of decisions must be 
made at a project level and the role of the Flight Dynamics team and other engineering organizations is 
to provide accurate, relevant and digestable data to aid in the decision. 

2. Re-allocate the currency : A mathematical example of this process is illustrated in the right two columns 
of the table and the equations which were applied are derieved in the appendix from the straight forward 
assumptions that the margin must be balanced with zero change in mass, and that the margins are equal 
on all requirements after balancing. 

3. Apply management reserve : The assumption that the mass change must be zero assumes that no mass 
growth is allowed. As the design matures, management may opt to release mass margin in the form of 
reduced weight growth allowance or reduced management reserve. This will allow improved 
performance across the flight regimes in the table, but the margin balancing principals remain the same. 

4. A combination of the above : Systems engineers may elect to modify requirements, re-allocate mass and 
apply reserve. 

The Orion systems engineering problem is much more complex that the example presented. There are many 
more performance metrics, precise sensitivities are often difficult to determine, and mass on the SM is not 
equivalent to mass on the CM since they drive different performance metrics. In practice, systems engineers 
combine classical margin balancing with engineering judgement, but systems engineers should always know their 
margins, understand sensitivities, and ensure that no margins stand out. 

The last of these is the crux of the vehicle-level systems engineering role. The role of the flight dyanamics 
analyst, on the other hand, is to provide data on margins and sensitivities. The next section discusses some lessons 
gleaned from these and other flight dyanamics studies which were conducted with the intent of supplying data to 
make vehicle-level decisions. 

B. Use of Time Domain Simulation and Monte Carlo Analysis in Trade Studies 

All of the trade studies described above used time domain simulation to produce the trade results. Time domain 
simulations techniques have become more and more popular over the last few decades to generate data for vehicle 
design. Monte Carlo analysis is often used to derive statistical correlations and sensitivities as shown above. This 
approach has some benefits, including: 
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• The process of simulation model development and integration often provides experience and insight to 
the developer into the governing physics and performance drivers in applicable flight phases (but this 
may not always be true for the analyst who may not have been involved in model development) 

• Modern computer systems are capable of handling hundreds of parameter variations over thousands of 
simulation runs. These can provide accurate Monte Carlo statistics over the parameter ranges used for 
the models used 

On Orion, Monte Carlo analysis will be used extensively for requirements verification, but it has also been used 
for requirements development, sensitivity analysis, and vehicle-level design. Design recommendations gleaned 
from these applications include: 

• Understand the underlying physics 

• Use Monte Carlo (or other statistical approaches) sparingly in requirements allocation or error 
budgeting 

• Understand when worst-case analysis is conservative and when it is appropriate 

• Avoid pitfalls in Monte Carlo analysis for requirements verification 

The following paragraphs details these recommendations. 

Recommendation: Understand the underlying physics 

The first recommendation is well known, but bears repeating. When new, inexperienced engineers join a 
project, they are often assigned simulation analysis tasks. These “turn the crank” jobs can allow engineers to begin 
producing results soon after joining an organization. However, it takes discipline on the part of supervisory 
personnel to make sure that the simulations are configured correctly, that the statistical attributes of Monte Carlo 
parameters are properly set, that output statistics are correctly interpreted, and that the analyst develops a knowledge 
of the underlying physics. We will discuss some of the statistical aspects below, but the system knowledge can only 
be gained by accompanying the simulation study with theoretical analysis. One rule of thumb: “never be surprised 
by a simulation result.” That is to say that the simulation or Monte Carlo analysis should confirm what the analyst 
expects or should refine the analysts’ estimate. When this is not the case, the analyst must understand why the 
results do not match expectations. The sim can be wrong! 

Recommendation: Use Monte Carlo sparingly in requirements allocation or error budgeting 

Figure 15 is a simple diagram of the Monte Carlo process which will aid in discussing lessons learned. In the 
process, the statistical properties of the input parameters are defined, then random values are selected for each of the 
parameters, and a simulation (or other process) is executed using these values. The resulting parameters of interest 
are recorded, and the process repeated many times. The statistical properties of the output provides information that 
may be used to decompose requirements, serve as an input for other design processes or to verify existing 
requirements. 

As an example of requirements decomposition, consider the heading error example in the roll orientation trade 
above. For this example the requirement on the GN&C substystem was “three times the standard deviation of the 

heading error (assuming zero mean) at 40 ft/sec groundspeed 
shall be less than 20°. ” One of the jobs of the Guidance, 
Navigation and Control (GN&C) system designer is to 
decompose this requirement into navigation accuracy and 
control error allocations. In this example the total error is 
just the sum of the navigation error and the control error. 

Figure 15. Monte Carlo Process Here, control error includes error due to disturbances - such 

as wind gusts wind shear. 

One familiar approach to allocating error to navigation and control is the worst case approach. The worst case 
may be defined as a hard limit, or it may be defined by a project policy such as “design to three sigma.” Either way, 
the total error is the sum of the worst case navigation and control errors. If the total error meets the requirement, 
then a portion of the margin may be allocated to the lower level requirements in proportion to their contribution. 
This method is the most conservative method of performing allocation since total error is highly unlikely to exceed 
the sum of the worst case errors. It is also the simplest method. 

If the total error does not meet the requirement, then the systems engineer may elect to petition for a more 
generous requirement (at a structural mass cost), try to improve navigation and or control performance (perhaps at a 
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Region of input space 


cost of more expensive hardware or more complex software) - or he may elect to use a less conservative allocation 
process. One method would be to assume that the total error distribution is a statistical combination of the input 
distributions. If both navigation and control error are random with Gaussian distribution, then the distribution of the 
sum of the errors is Gaussian with standard deviation equal to the root-mean-square (RMS) of the inputs. 

If the Gaussian statistical assumptions are not applicable to the inputs, or if the relationship between inputs and 
outputs is complex, then, Monte Carlo analysis is sometimes used to 
relate the statistical properties of the inputs to the outputs. However, if 
Monte Carlo is used, two pitfalls must be avoided: 

• The output distribution must be close enough to 
Gaussian to meet the intent of the requirement 

• Margin must be applied during requirements allocation 

The later pitfall is the most common and causes the most difficulty. 

If the lower level requirements (navigation and control error 
specifications in the example) are simply left at their expected “3 a” 
values, and if there is insufficient margin at the next level (total heading 
error in the example), then any change in applicable parameters or 
assumptions may cause an increase in navigation or control variances, 
which may in turn cause an exceedance of the heading error 
requirement, which may necessitate 
redefinition of the requirement, which can 
affect structural mass and ultimately require re- 
allocation of mass at the vehicle level. In our 
example, if the landing site changes, and the 
wind shear statistics worsen, then the control 
variances will increase, and the effect can 
propagate up the requirements chain. To 
prevent this, margin should be consciously 

allocated early through all requirements levels. ^ _ 

Figure 17. Driving Parameters 

Recommendation: Understand when worst-case analysis is conservative and when it is appropriate 



pi 


Figure 16. Parameter Box 



As mentioned above, worst case analysis is a simple approach to requirement decomposition and includes a level 
of conservatism or “built in” margin. But analysts often confuse the low probabilityof a worst case combination 
with conservatism. To illustrate this, consider Figure 16 - an abstraction of the input parameter space. The 
horizontal axis represents the value of an input parameter, “pi” and the vertical axis another parameter, “p2.” The 
mean values of pi and p2 are shown by the cross at center, while the red box represents the range of values between 
mean minus 3a and mean plus 3a. The green ellipse represents the region in the pi, p2 parameter space for which a 
requirement is met. 

The common argument for using a statistical approach to reduce conservatism is to note that the probablitly of 
the pi, p2 pair existing near the corners of the box is low. [Include equation here. I think it’s (l-99.73) A n]. As the 
number of parameters increases, the probability of parameters combining at the corners of the box in a “worst on 
worst” combination rapidly approaches zero. So assessing to the corners of the box that cause requirements 
exceedance may be unnecessarily conservative. 

While it is true that worst case analysis assumes a low probability condition, the degree to which it is 
conservative depends on the relative sensitivity of the output to the parameters. The left diagram of Figure 17 shows 
a case where pi is a “driving” parameter and the output is relatively insensitive to p2. In this case, evaluating the 
corner of the box does not add a significant amount of conservatism - even though the pi, p2 pair at the corner is 
highly improbable. And the same is true for the right hand diagram, in which the output is not highly sensitive to 
either parameter in the context of their variances. 

It is often the case that a system response is driven largely by one or two parameters, so, depending on the 
amount of margin available, and the relative sensitivity of the output to the other parameters, a worst case, or 
“corners of the box” approach to requirement decomposition is often the simplest and most robust method. 

This worst case approach was used by the Orion flight dynamics team in defining the requirements for CG offset 
as discussed in the L/D section above. For missions to the International Space Station, the CG offset was derived by 
assuming that aerodynamic parameters were at their worst values (when uniformly distributed) or their 3 a values 
when Gaussian. This allowed a simple definition for a CG box with adequate margin for uncertainties and 
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operational variations in CG. For the Lunar mission, however, margin was much more constrained - largely by a 
higher minimum L/D requirement, so a more sophisticated statistical approach was taken, as described in reference 
XXX. 

Recommendation: Avoid pitfalls in Monte Carlo analysis for requirements verification 

As the project approaches CDR, the requirements allocation process is largely complete, and the focus shifts 
toward proving requirements compliance via pre-CDR verification. During this phase, simulation analysis using 
higher fidelity models is an appropriate method for assessing requirements compliance. However, several common 
pitfalls should be avoided: 

1 . Set parameters according to bounding requirements - not expected values 

2. Margin is determined from output results - not input variations 

3. ... however, input variations can and should be used to determine robustness - find out where it breaks 

4. TBS 

Conclusions 

TBS 


Appendix 


A. Margin Balancing Equations 

Once margin and sensitivity data are available for each requirement, the equations to re-allocate the independent 
cost variable (often mass, or monetary cost, or a combination) may be derived as follows: 

The change in margin for the i th requirement, A M t is added to the existing margin to achieve the balanced margin 
which we desire to be constant for all requirements: 


The change in margin may be replaced by the change in cost variable, C times the sensitivity of margin to the 
cost variable: 

( 1 ) 

so 

and 

( 2 ) 

Now, when re-allocating cost, the net cost change between all n requirements must be zero: 

= 0 
so 

= 0 

This expression contains the constant balanced margin for each requirement, M b , and the know values for the 
margins and sensitivities for each requirement. To solve for M b , the common denominator for each of the terms in 
the sum is multiplied by each sum: 


Define: 


so 


so the desired balanced margin value is: 
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( 3 ) 


and the changes in cost variable and margin for each requirement are may be determined from 1, and 2 above. 


B. Margin for Statistical Success Requirements 

The approach taken in this exercise was to presume that abort failure is the result of an “underlying performance 
parameter” that exceeds a limit. We further assume that the underlying parameter has a Gaussian distribution of 
mean zero and a standard deviation that causes the limit to be exceeded 5% of the time. This standard deviation 
then is the required standard deviation of the parameter. We then compute the standard deviation that causes the 
parameter to exceed the limit at the observed percentage. Margin is then computed as the difference between these 
standard deviations. 

TBS. 
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